1. Field of the Invention
This invention relates generally to computer input devices, and more particularly provides a system and method for computer input of dynamic mental information such as visual mental imagery.
2. Description of the Background Art
Conventional computer input devices include keyboards, mice, tracker balls, touch sensitive displays and microphones. Each of these input devices translate physical actions by a user into computer instructions. For example, a computer may recognize mouse movement as a computer instruction to move a pointer and may recognize key depression on a keyboard as a computer instruction to generate text.
For physically disabled individuals who cannot control a mouse, type on a keyboard or speak into a microphone, computer input is difficult and potentially impossible. There have been several studies into the use of bioelectrical activity in the brain to control a pointer device. An article by John Charles, entitled xe2x80x9cNeural Interfaces Link the Mind and the Machine,xe2x80x9d indicates that, by recognizing particular electrical biosignals such as electroencephalograph (EEG), electrooculograph (EOG) and electromyograph (EMG), it is possible to move a computer pointer. However, this technique does not enable computer input of text, images, sound, body movement or other sensory information to a computer.
Therefore, a system and method enabling computer input of dynamic mental information are needed.
Images on the retina are geometric mappings (projections) of what a person sees. These images are carried to a region in the visual cortex commonly referred to as the V1 region (or the primary visual cortex). The V1 region is retinotopically mapped, i.e., the physical locations of the activated neurons in the V1 region are a geometric mapping (homeomorphism) of the image on the retina. The image in the V1 region can be and has been read by using brain-scanning instruments such as functional magnetic resonance imaging (functional MRI) or positron emission tomography (PET). Neurons then carry the signals out of the V1 region and into deeper regions of the brain, which are not geometrically mapped. It has been recently recognized that there is feedback from those deeper regions back to the V1 region. It has also been recently recognized that this feedback includes images generated by the imagination. Accordingly, a system embodying the present invention reads these feedback signals to obtain and interpret this dynamic mental imagery as computer input.
It will be appreciated that other brain regions (e.g., the auditory cortex, the somatosensory cortex, etc.) may similarly provide physiological responses to other actual sensory information (e.g., sounds and voices, body movement, etc.) and may similarly receive feedback to other imagined sensory information (e.g., imagined sounds and voices, imagined body movement, etc.). Preferably, the other brain regions are large enough to distinguish content, are mapped according to a continuous sensory quantity, and receive feedback corresponding to that quantity. Accordingly, all sensory modalities could be used together to control a virtual reality system.
The system of the present invention calibrates a user""s brain region (e.g., the primary visual cortex or V1 region) to actual sensory information (e.g., the visual field), and enables imagined sensory information (e.g., dynamic mental imagery) to be interpreted as computer input. The system comprises a configuration engine and an input device control engine. The configuration engine includes a test pattern; a functional information gatherer for presenting the test pattern to a user; a brain-scanning device interface for obtaining functional information from a region in the user""s brain that provides a physiological response to the test pattern and that receives feedback corresponding to imagined sensory information; and a mapping engine for using the functional information to map the user""s brain region to the test pattern. The input device control engine includes a brain-scanning device interface for obtaining functional information from a brain region that provides a physiological response to actual sensory information and that receives feedback corresponding to imagined sensory information; an interpretation engine for interpreting the feedback; and a computer control engine for using the interpreted feedback as computer input.
The present invention further provides a method for calibrating a user""s brain region to actual sensory information, and enables imagined sensory information to be interpreted as computer input. The method comprises a configuration process and a computer input process. The configuration process includes presenting a test pattern to a user; obtaining functional information from a region in the user""s brain that provides a physiological response to the test pattern and that receives feedback corresponding to imagined sensory information; and using the functional information to map the user""s brain region to the test pattern. The computer input process includes obtaining functional information from a brain region that provides a physiological response to actual sensory information and that receives feedback corresponding to imagined sensory information; interpreting the feedback; and using the interpreted feedback as computer input.
The system and method of the present invention may advantageously enable computer input of imagined sensory information such as imagined images, imagined sounds, imagined body movements, etc. Accordingly, an individual unable to manipulate conventional computer input devices may be able to control a computer using only thoughts and imagination. Further, computer input of imagined sensory information may be faster than traditional computer input. Still further, since computer input is effected via the imagination, drawing ability is not as crucial.